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Abstract. During the past two years, many sensors have been tested in an open-loop fashion in the Marshall Space 
Flight Center (MSFC) Flight Robotics Laboratory (FRL) to both determine their suitability for use in Automated 
Rendezvous and Docking (AR&D) systems and to ensure the test facility is prepared for future multi-sensor testing. 
The primary focus of this work was in support of the CEV AR&D system, because the AR&D sensor technology area 
was identified as one of the top risks in the program. In 2006, four different sensors were tested individually or in a 
pair in the MSFC FRL. In 2007, four sensors, two each of two different types, were tested simultaneously. In each set 
of tests, the target was moved through a series of pre-planned trajectories while the sensor tracked it. In addition, a 
laser tracker “truth” sensor also measured the target motion. The tests demonstrated the functionality of testing four 
sensors simultaneously as well as the capabilities (both good and bad) of all of the different sensors tested. This paper 
outlines the test setup and conditions, briefly describes the facility, summarizes the earlier results of the individual 
sensor tests, and describes in some detail the results of the four-sensor testing. Post-test analysis includes data fusion 
by minimum variance estimation and sequential Kalman filtering. This Sensor Technology Project work was funded 
by NASA’s Exploration Technology Development Program. 
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INTRODUCTION 

The Sensor Technology Program is one of the many projects that has been funded by NASA’s Exploration 
Technology Development Program in the last few years. The Sensor Technology Program (STP) was formed in part 
to help NASA become more knowledgeable about the different sensor technologies and implementations available 
for use in Automated Rendezvous and Docking (AR&D) systems. In addition to sensor knowledge, the Sensor 
Technology Program was charged with finding facilities and methods that could adequately test the performance of 
space sensors, particularly sensors of a type that may be used for the Crew Exploration Vehicle (CEV). The work at 
the Marshall Space Flight Center (MSFC) in 2007 focused on a series of multi-sensor testing tasks. The work 
required the development of test plans, test facility requirements, target hardware, and sensor mounts. Four sensors 
were used in this test: two MSFC Advanced Video Guidance Sensors (AVGS) and two laser rangefinders. The 
AVGS was a laser-based system that tracks a set of reflectors on the target vehicle, and the laser rangefinder was an 
inexpensive commercial off-the-shelf sensor that continuously measured range. 

Sensor characterization and testing for this group of four sensors was conducted at MSFC’s Flight Robotics 
Laboratory (FRL) using the 6-DOF gantry system, called the Dynamic Overhead Target Simulator (DOTS). The 
target vehicle used for “docking” in the laboratory was a mockup that was representative of a potential CEV docking 
system (essentially an International Space Station [ISS] docking port) with added retro-reflectors for the sensors. 
Figure 1 is a picture of the mockup with the retro-reflective targets visible. The target was moved through a series 
of trajectories while the four sensors and two truth sensors collected data. The overall test process was similar to 
that performed on several sensors individually in 2006, and that testing is detailed in a previous conference paper 
(Mitchell, et al., 2007). The purposes of this set of testing were to ensure that multiple sensors would not interfere 



with each other and to demonstrate that simultaneous sensor testing could be conducted, with each sensor’s data 
time stamped for post-processing with respect to the truth data. The data processing involved converting all of the 
different sensor data into a single coordinate system, synchronizing the time stamps, and analyzing the noise and 
errors of the sensor data with respect to the truth data. In addition, two different approaches to blending the four 
sensors’ data were applied in the post-processing. 



(a) Target mockup: Retro-reflectors are visible on the right and in the SRTs. (b) Sensor setup. 

FIGURE 1 . Target Mockup and Sensor Layout in Flight Robotics Laboratory. 

TESTING 

The test facility, the FRL, had been used the prior year to conduct similar testing, so minimal work was required on 
the part of the facility. This year’s work involved modifying the previous procedure to accommodate the new round 
of tests and remounting the target mockup on the DOTS. The target mockup required some modification in the form 
of additional targets. The previous testing required two targets for the AVGS: a Long Range Target (LRT) and a 
Short Range Target (SRT). The LRT is used by the AVGS for ranges greater than 8 meters, and the SRT is used by 
the AVGS for ranges from 25 meters down to 1 me.ter. The new test configuration involved two AVGS units, in 
which the sensors’ fields-of-view (FOV) were each +/- 8 degrees, and the sensor separation was approximately 1 
meter. Hence two different SRTs were required, one for each sensor. The two different SRTs, shown in Fig. la, 
were mounted to line up with the two different AVGS units when the target mockup was one meter from the 
sensors, while the LRT, the three reflectors to the right of the target mockup in Fig, la, was shared by both sensors. 

Sensors 

Two different AVGS units were used for this testing: the Demonstration of Autonomous Rendezvous Technologies 
(DART) flight-like Serial Number 2 (SN2) unit as well as one of the DART final prototypes (FP2). The FP2 unit 
was built as a prototype to ensure that the engineering,, design, and performance were sound. The SN2 unit was built 
to flight specifications, but was used for various software, EMI, Thermal Vacuum, and performance tests. The SN2 





unit is the topmost box (light box with dark lid) in Fig. lb, while the FP2 unit is the box on the bottom right of Fig. 
lb. The FP2 unit did not quite line up with its target due to mounting issues, so the offset had to be corrected in the 
software by adjusting the target matrix. 

The software that was used in the AVGS units for the STP testing was a variation of the software developed for the 
Orbital Express (OE) AVGS, with such changes as were necessary to work with the new target configuration. The 
new target configuration changed both the shape of the LRT, the relative position of the SRT to the LRT, and the 
number of SRTs that could be in the FOV. After modification, the new software and associated target matrix 
information had to be loaded onto the two different AVGS units and tested both separately and together prior to the 
runs for record. The SRTs in Fig. la are rotated 180 degrees from one another in order to provide the differentiation 
needed in the sensor software to ensure that neither sensor would track the other’s SRT. 

One potential concern about the AVGS performance was that the OE AVGS, which was not available for this 
testing, had some hardware changes as well as software changes. Two of the improvements from the DART AVGS 
to the OE AVGS were in the lasers - they were more tightly temperature-controlled for OE (+/- 3 degrees C rather 
than +/- 20 degrees C) and their output was more evenly distributed across the FOV (1/2 power in the comers rather 
than 1/8 power in the comers). Also, a flight-like SRT was not available, so two targets were built from available 
filters, comer cubes, and aluminum. The newly-built SRTs did not have anti-reflection coatings, which may have 
affected their performance at close range. At close range, the front surface of the filters can reflect both lasers’ light 
straight back into the sensor, causing the spot centroids to shift and thus interfering with the solution. The anti- 
reflection coating prevents this phenomenon. The AVGS units’ desired performance was based on the OE 
specification of +/- 1.2 cm at 1-3 meters, and +/- 15 cm at ranges greater than 5 meters. These extra DART AVGS 
units were not expected to meet those specifications, but they were expected to be fairly close. 

The other two sensors used in this testing were laser rangefinders. These particular commercial laser rangefinders 
were chosen because they covered the required range of operations (1 to 40 meters), provided continuous range 
output, and were inexpensive. The specifications for the laser rangefinders were fairly broad - an accuracy of +/- 
one meter and a resolution of +/- 0.2 meters. They were intended to simulate the reduced accuracy of long-range 
sensors at close ranges. 

Tests Conducted 

The sensors were tested using two categories of open-loop test trajectories: (1) sensor characterization trajectories 
designed to test a wide range of performance parameters, and (2) CEV-specific trajectories designed to test 
performance during CEV-like approach and departure profiles. A total of 35 different trajectories were run. In each 
case, the AVGS and laser rangefinder sensor data was collected along with “truth” data from the DOTS encoders. 
Also, for most trajectories, more accurate truth data was collected using a Leica laser tracker metrology system. 

Expected Results 

The AVGS units were expected to track the Long Range Target (LRT) and the Short Range Target (SRT) whenever 
they were in the sensor’s field-of-view (FOV) and whenever they were within their appropriate range parameters 
(SRT from 1 meter out to about 25 meters, LRT beyond 8 meters.) The accuracy was expected to be fairly good, but 
not as good as the Orbital Express (OE) version of the AVGS. The OE AVGS had a more evenly distributed spread 
for the lasers, more tightly controlled imager and laser temperatures, and a target with anti-reflection coatings. The 
laser rangefinders were expected to track the target whenever it was directly in front of the rangefinders, and they 
were expected to give a large constant value (the distance to the back wall of the facility) whenever the target was 
not directly in front of the rangefinders. 


ACTUAL RESULTS 

A great deal of data was collected. The truth data was collected at high data rates to allow close comparison with 
the slower sensor data. The Leica data was collected at 100 Hz, and the DOTS data was collected at 50Hz. AVGS 
data was collected at 5 Hz, and the laser rangefinder data was collected at about 20 Hz. A sample plot showing the 
X-axis (essentially the range) data for the truth data (DOTS and Leica) and both AVGS unit’s LRT and SRT data is 



shown in Fig. 2. Notice that all of the different data sources generated similar outputs. In this particular trajectory, 
the target was moved back and forth across the facility while holding the range and elevation constant for one 
direction of travel. The AVGS data dropouts correlate with periods of time when the target was moved outside the 
sensor’s FOV. 

All of the sensor data was converted to a common coordinate frame. Both AVGS units’ LRT and SRT data was 
compared with the Leica and DOTS data and checked for bias, noise, and completeness (i.e. did the sensor and test 
facility meet test requirements). The laser rangefinder data was also checked for completeness and was compared 
with the X-axis of the truth data for a general test of the “goodness” of the data. In all cases, AVGS sensor data 
corresponded fairly well with truth data. The laser rangefinder data was less useful, partially because of its low 
resolution and partially because the units internally averaged several readings per output. The averaging caused data 
to gradually move from accurate to maxed-out rather than jumping to the maximum value as was expected. For 
simple processing, X-axis error data that differed from the truth data by more than 2 meters was ignored in 
generating statistics. For the blended solutions discussed below, other methods were used to deal with bad data. 
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FIGURE 2. Range Plot of a Trajectory Showing the Two Truth Data Sensors and All Data from the AVGS Units. 

Blended Solutions 

The multi-sensor testing of two AVGS units and two laser rangefinders, all tracking the trajectory of a common 
target platform, suggests an integrated solution. The target platform moves along a prescribed trajectory, realized by 
the eight degree-of-ffeedom (DOF) robotic manipulator, DOTS, in MSFC’s Flight Robotics Lab. A Kalman filter 
approach to integrating the multi-sensor data is appropriate for state estimation of a dynamical system, but 
traditional approaches require a valid state-space model for the platform dynamics. Neither a dynamical model for 
the DOTS test manipulator, nor a model representing the dynamics of a spacecraft, is appropriate for the trajectories 
in these tests. Hence a sequential minimum variance estimate of the 6-DOF solution has been developed, as well as 
a Kalman filter approach using the minimal variance estimate for state estimation. The sequential minimum 


variance estimate should in theory be similar to the estimate from a sequential Kalman filter driven by a valid state 
model, and where the measurement error covariance matrix is used as the weighting matrix (Junkins, 1978). 

Minimum Variance Estimation 

The sequential minimum variance estimator processes measurement data Y at a specific time t ky represented as Y k . 
The measurements are related to the position and attitude states X k at time t k by Y k = A h X k + V k , where V k is the 
vector of measurement errors, with covariance matrix = E(V k V k ) . The estimate of the six states at time t k+1 is 

Xk+\ = X k +K k+l (Y k+l — A k+l X k ) , (1) 

in which the gain K k+} is K M = P k+ \A k+l W k+l , defined by the state covariance matrix P k+lt and where the weight 
matrix W M is the inverse of the measurement error covariance matrix A k+1 . The state covariance matrix is given by 

( 2 ) 

Since the measurements from several sensors need to be available at a given time t k , the sensor data has been 
interpolated where necessary. The interpolation algorithm used was Matlab’s PCHIP, Piecewise Cubic Hermite 
Interpolating Polynomial, which forms shape-preserving piecewise cubic Hermite interpolants in a cubic spline, and 
is suitable for data that is not smooth. Data from the two AVGS Short Range Target (SRT) solutions and the Long 
Range Target (LRT) solutions, and the two laser rangefinders (LRF), when available, were interpolated to time steps 
at 5 Hz. The “truth” data from the Leica and the DOTS position estimates was also interpolated at this 5 Hz rate. 


TABLE 1. Single-Sensor Error-Model Coefficients. 



SN2 SRT 



SN2 LRT 



Range-Coeff 

Bias 


Range-Coeff 

Bias 

X(ft) 

0.0040 

-0.1948 

X (ft) 

0.0025 

-0.1883 

Y (ft) 

0.0013 

0.0620 

Y (ft) 

- 0.0000 

0.11164 

Z(ft) 

0.0012 

-0.0558 

Z(ft) 

0.0022 

-0.0893 

Pitch (deg) 

-0.0079 

1.4946 

Pitch (deg) 

-0.0044 

1.2240 

Yaw (deg) 

-0.0339 

0.4688 

Yaw (deg) 

-0.0048 

0.4666 

Roll (deg) 

-0.0033 

1.6998 

Roll (deg) 

-0.0081 

2.1661 


FP2 SRT 



FP2 LRT 



Range-Coeff 

Bias 


Range-Coeff 

Bias 

X(ft) 

0.0099 

-0.4512 

x(ft) 

0.0022 

-0.1362 

Y (ft) 

-0.0053 

0.2074 

Y (ft) 

-0.0046 

0.1705 

2 (ft) 

-0.0037 

-0.0994 

z(ft) 

-0.0043 

-0.0570 

Pitch (deg) 

0.0101 

1.0586 

Pitch (deg) 

-0.0064 

1.1262 

Yaw (deg) 

0.0066 

0.5100 

Yaw (deg) 

-0.0073 

0.9873 

Roll (deg) 

0.0064 

1.3994 

Roll (deg) 

-0.0063 

2.6277 


LRF1 



LRF2 



Range-Coeff 

Bias 


Range-Coeff 

Bias 

X(ft) 

0.0836 

-4.8658 

x(ft) 

0.1569 

4.1727 


This minimum variance estimate algorithm needs a measurement error covariance matrix A , the inverse of the 
weighting matrix A* = W k 1 . In this implementation the measurement error covariance matrix was maintained as a 

constant for each trajectory. The variances were set by a priori determination of the standard deviation in the error 
between each sensor’s solution and the “truth” data from either the Leica or the DOTS. In addition, since the 
position and attitude are changing at each time step but are not being driven by dynamic state equations, the state 

estimate X k was set to zero in estimating X k+l , so that each state estimate is not influenced by the solution at the 



previous time step. Equation (1) then becomes the Gauss-Markov Theorem formulation of minimum variance 
estimation. ' 

To build an error model for each sensor, the error between the DOTS or Leica truth data and the solution from each 
sensor was determined. The error was fit as a straight line using linear least squares, using range as the independent 
variable. The coefficients for the linear fit were stored for each of the six degrees-of-freedom, and for each of the 
sensors, to be used in both the minimum variance blended estimate and the Kalman filter blended estimate. The 
dominant coefficients in these fits are biases, representing mounting misalignments and testing misalignments. The 
error model coefficients, which were determined by analyzing several sets of test data, are listed in Table 1 . 

The blended minimum variance estimate from several sensors was examined by using just the two AVGS short- 
range solutions, using the two AVGS short-range and the two AVGS long-range solutions, and using the two AVGS 
short- and long-range solutions with the two laser rangefinder solutions. When one or more sensors did not have 
data at a particular time step, the estimator used only the available data. 

CEV_trajectory3 is used in this paper for comparisons of the minimum variance estimate blended solution. A 
comparison of the truth data from DOTS and the blended solution shows good agreement; the difference between 
the truth data and the blended solution showed errors that were less than 0.1 ft in position and approximately 0.1 -0.2 
degrees in attitude. Errors from the SRT solutions at longer ranges are dominant when they begin contributing to the 
blended solution. The blended solution from the two AVGS units is indistinguishable from the solution with the 
addition of the laser rangefinders, indicating that the contribution from the laser rangefinders is insignificant in 
improving the blended solution. It also shows that the LRF measurements do not degrade the blended solution, due 
to appropriate weighting conditions. 

The error contributions from the SRT solutions at longer ranges suggest that eliminating SRT measurements greater 
than 60 ft might reduce the errors. The SRT contributions were restricted to below a threshold of 62 ft, and the LRT 
contributions to above 60 ft. However, the errors are actually larger using thresholds, particularly at the 60-ft 
transition from the LRT to the SRT solution. Hence thresholding is not used in the blended solutions. 

Fig. 3 shows the number of inputs used in the blended solution, and indicates when certain targets were either 
blocked or outside a sensor’s FOV. Six sensors indicate that both SRT and LRT solutions from the two AVGS units, 
and the two laser rangefinders, contributed to the blended solution. Spikes from three sensors to four sensors 
usually represent the addition of a laser rangefinder reading to the solution with two SRT solutions and one LRT 
solution. The laser rangefinder contributions were pre-filtered to eliminate readings from the back wall of the FRL. 
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FIGURE 3. Number of Sensors Used in Minimum Variance Estimate. 
Kalman Filter Estimates 

A sequential linear Kalman filter was also developed for data fusion. The state model is given by 


( 3 ) 



where W k is the process noise with covariance matrix Q k . Since the trajectories being followed in this testing do not 
follow a vehicle dynamics model, we use the minimum variance estimate at that time step as the initial state 

estimate, and set the state transition matrix O* to the identity matrix. The process noise covariance matrix Q k is 
assumed to be zero for these test trajectories. The state covariance estimate is propagated forward by 

= ®k-\Pk-fik-\ Yk-\Qk-\Yk-\ * W 


and the Kalman gain is updated by 


K.=flT4wr4 +A,I 


-1 


( 5 ) 


The state is updated by Equation (1), rewritten as 

x; = x- k +K k v k -A k x n, 

and the state covariance matrix is updated by 

p;=\i-K k A k ]p;. 


( 6 ) 

( 7 ) 


The sequential linear Kalman filter was used to estimate positions and attitudes from the two AVGS SRT solutions 
and the two AVGS LRT solutions. The results are identical to the minimum variance estimate. Artifacts in the 
minimum variance estimate occur at approximately 450 seconds, as shown in Fig. 4a, which are brought into the 
blended solution from the FP2 LRT measurements. To remove these artifacts, a Chi-squared test for each of the 
sensor measurements was added. This test determines suitability of the measurement before being used in the 
Kalman filter estimate. The Chi-squared test is determined by 

^,2 = &k \ € k ^ 


where £ k = Y k — A k X k is the innovation, or the difference between the actual measurements and the expected 

measurements, and n is the length of the measurement vector. The Chi-squared test is done separately for the 
position and attitude vectors, so n is 3. Any measurement whose Chi-square is greater than a specified threshold is 
not used in the Kalman estimate. A Chi-squared threshold of 20 was used in this analysis; a threshold of 1 0 
eliminated most of the multi-sensor blending, relying on only one sensor for most time steps. The yaw estimate 
from the Kalman filter without Chi-squared testing is shown in Fig. 4a, and with the Chi-squared test in Fig. 4b. As 
expected, the artifacts at LRT-to-SRT transitions are removed with Chi-squared testing. 
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(a) Yaw Estimate in degrees, without Chi-squared Test. 




FIGURE 4. Kalman Filter Estimate for Yaw, With and Without Chi-squared Test. 


Summary of Blended Solutions 


Data collected from the multi-sensor tests was processed through both a sequential minimum variance estimator and 
a linear sequential Kalman filter to produce blended solutions. Setting thresholds to eliminate noisy measurements 
from specified sensor regimes was found to produce larger errors than just using the minimum variance estimator 
with appropriate standard deviations. An error model for test misalignments, which modeled error as a bias plus a 
constant coefficient of range, was developed from several test data sets, for each of the sensors. This same error 
model was applied to all processed data. Contributions from the two laser rangefinders were insignificant in the 
blended solutions, but did not degrade the solution. 

A Chi-squared test for each of the sensors was made at each measurement interval, to determine if that measurement 
was appropriate for use in the Kalman filter. This test was able to remove many artifacts that appeared in the 
blended solutions due to mis-measurements from sensor FP2. However, it was not successful in removing all of 
these artifacts. 

Additional work could be done in developing an error model for each sensor. The simple linear model of error as a 
function of range removed much of the errors, but could be improved by modeling errors that appear to be the result 
of spherical aberration in some sensors. 

ANOMALIES 

There were few anomalies observed during this series of tests. The most annoying of the anomalies was that there 
was a significant roll bias between the truth data and the sensor data that was not noticed until well into the testing. 
The method by which the DOTS calibration was checked each morning was being performed on the wrong portion 
of the target mockup, causing the data to be biased. This bias was easily corrected after the testing was concluded. 

LESSONS LEARNED 

One lesson was identified during the last series of tests, but could not be applied due to the desire to keep the 
trajectories unchanged. The Leica truth sensor was limited in its maximum and minimum range, but some of the 
trajectories did not take that limitation into account three -dimensionally (i.e. the range used in trajectory design was 
approximated by motion only in the X-axis.) Unfortunately, for some trajectories, there was a short dropout in the 
Leica truth data at extremes of range. This created a post-processing problem, but not very much data was lost (a 
few seconds out of some trajectories that took 20 to 30 minutes to run.) 

CONCLUSION 

This Sensor Technology Program multi-sensor testing proved that MSFC’s FRL can provide the capability to test 
four proximity operations sensors simultaneously, while gathering time-stamped data from them and from the truth 
sensors. In addition, no interference was seen between any of the sensors. Finally, the sensor fusion efforts were 
quite successful in creating a blended solution that was better than any one sensor’s data. Overall, this testing was 
quite successful. 
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